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Executive summary 



Introduction and methods 

This summary presents the key findings from an independent evaluation, 
carried out in the school year 2005-6 by the National Foundation for 
Educational Research (NFER), of the implementation of Renaissance 
Learning mathematics and reading programs in selected Specialist and feeder 
schools in London. The research was conducted within a quasi-experimental 
framework which involved matching schools implementing the programmes 
(‘treatment’ schools) with similar schools where the program was not being 
implemented (‘comparison’ schools). In order to evaluate the impact of the 
programs the following three-fold methodology was used: 

• standardised tests administered in two sweeps 

• questionnaire surveys to assess pupils attitudes and motivation, also in two 
sweeps 

• face-to-face interviews with teachers responsible for implementing the 
programs. 

Trends in pupil attainment 

Standardised tests were carried out at two points, one early on in the school 
year (November 2005) and another later in the year (June- July 2006). This 
pre-test post-test approach allowed comparisons over time (approximately an 
eight-month period) and across program and non-program schools. It should 
be stressed that, although the test data can be seen as being indicative of the 
general progress made (or not made) by the pupils, it may not be possible to 
attribute causality to the Renaissance programs. Sufficient test papers for 
analysis were returned from 21 schools: 14 treatment and seven comparison 
schools. The main findings regarding average standardised test scores can be 
summarised as follows: 

• Mathematics in primary schools: average standardised test scores for 
mathematics improved in two out of four treatment schools; in the three 
comparison schools, one saw a decline in average score and two saw 
improvements. 

• Mathematics in secondary schools: two treatment secondary schools 
returned their tests, in both cases the average standardised score improved 
from sweep 1 to sweep 2. One comparison secondary school returned their 
mathematics tests: the average standardised score in this school also 
improved , but not to the extent that was evident in the treatment schools. 

• Reading in primary schools: average standardised test scores for reading 
improved in five out of six treatment schools; in the two comparison 
schools, one saw a decline in average score and one saw an improvement. 
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• Reading in secondary schools: two treatment secondary schools returned 
their tests, in both cases the average standardised score declined from 
sweep 1 to sweep 2. One comparison secondary school returned their 
reading tests: the average standardised score in this school also declined. 



Based upon the average standardised scores, the key finding regarding pupil 
attainment was that the most significant improvements occurred in primary 
reading and secondary mathematics schools. In primary mathematics and 
secondary reading schools, the findings are less conclusive, though there are 
many reasons, unconnected with the use or non-use of the Renaissance 
programs, as to why this might be so. In addition to these broad trends, there 
are some individual schools which achieved notable successes in a short space 
of time: these successes are discussed in more detail in the full report. 

Survey findings: progress over time 

The Renaissance programs, through the use of regular built-in assessment and 
testing, enable pupils and teachers to see what progress is being made in 
attainment. The programs are also designed to meet individual learning 
requirements and to encourage and motivate pupils in reading and 
mathematics. For this reason, questionnaire surveys were issued to the same 
groups of pupils taking the standardised tests. 

The information collected from the surveys of pupils enabled the research 
team to identify any changes in attitudes, confidence or motivation in pupils in 
treatment schools over the academic year (the same questionnaire was 
completed twice), and also to compare the attitudes of pupils in program 
schools with those of pupils in non-program schools. Selected key findings 
from the surveys are presented below (complete figures for responses to 
individual questions are provided in the full report). All tables are based on 
sweep 1 and sweep 2 findings from pupils in treatment schools only. 

Mathematics in primary schools 

• Overall, there was little change in Year 4/5 pupils’ perceptions over time 
as to how well they did in mathematics, although there was a more positive 
view of their own ability in relation to others (12 per cent of pupils in 
sweep 1 saying that ‘Maths is harder for me than for any of my 
classmates’, going down to nine per cent) and more pupils disagreeing that 
they were not good at mathematics (51 per cent increased to 55 per cent). 

• Year 4/5 survey respondents expressed mixed feelings about their levels of 
enjoyment of recent mathematics lessons, with fewer pupils finding them 
‘very enjoyable’ (58 per cent decreased to 51 per cent), but more finding 
them ‘quite enjoyable’ (32 per cent increased to 38 per cent). 

• There were positive developments in attitudes to progress in mathematics, 
with more respondents believing that they had made a great deal of 
progress by the time of the sweep 2 survey (41 per cent increased to 48 per 
cent) and a fall in the percentage believing they had made ‘no progress’ 
(nine per cent decreased to five per cent). 
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Mathematics in secondary schools 

Secondary school pupils (Year 7) were asked to complete questionnaires on 
their attitudes to mathematics at the same time as the primary schools, but it 
should be noted that the response rate of those who completed both sweep 1 
and sweep 2 was much lower, with a total of 38 Year 7 pupils returning 
questionnaires. 

• The responses across time were broadly positive, with a particular increase 
in pupils who agreed ‘a lot’ that they did well in mathematics (from 50 per 
cent to 61 per cent). Year 7 pupils’ responses to the question of whether 
they enjoyed learning mathematics were more cautious: 47 per cent said 
that they enjoyed mathematics ‘a lot’ in sweep 1, but this decreased to 34 
per cent in sweep 2. 

• There was little change in Year 7 pupils’ views of enjoyment of lessons, 
with the great majority (82 per cent) still finding mathematics ‘quite’ or 
‘very’ enjoyable. 

• There was a marked increase in the percentage of secondary pupils who 
thought they had made ‘a great deal of progress’ in mathematics over the 
course of the year (from 42 per cent to 63 per cent). The improved test 
scores for secondary Accelerated Maths schools reported above would 
support this perception of good progress made. 



Reading in primary schools 

• Attitudes to reading, in both sweeps of the survey, were positive, with the 
majority of Year 4 and 5 pupils agreeing that they enjoyed reading (around 
three-quarters) and did not find it difficult (around four- fifths). However, 
there was a downward trend between the two surveys in the percentages 
that agreed that they liked reading stories (from 76 to 70 per cent) and 
generally enjoyed reading (from 70 to 64 per cent). 

• Whilst the percentage of pupils claiming to read every day decreased 
(from 30 to 22 per cent), there was an increase in those who said that they 
read most days (44 to 48 per cent) and no change in those who said that 
they did not often read at home. 

• The proportion of pupils who found reading ‘very enjoyable’ decreased 
from 49 to 44 per cent, though this is counterbalanced by the fact that the 
proportion reporting reading ‘quite enjoyable’ increased by the same 
margin. 

• Year 4/5 pupils’ views on progress in reading were positive, with an 
increase in those saying that they had made ‘a great deal’ of progress 
(from 46 per cent to 52 per cent). 

Reading in secondary schools 

In general Year 7 pupils’ attitudes towards reading seem to have become less 

positive over the course of the school year. For example, by sweep 2 of the 

questionnaire survey, Year 7 pupils: 

• were less likely to l ik e reading stories (74 per cent down to 65 per cent) 
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• still had the same likelihood of reading at home every day (17 per cent for 
both sweeps) 

• were less likely to find reading ‘very’ or ‘quite’ enjoyable (85 per cent 
down to 73 per cent) 

• less likely to report having made a great deal of progress (down from 43 
per cent to 34 per cent). 

It needs to be stressed, however, that the Year 7 pupils were still generally 
positive about reading: it was just that their enthusiasm appeared to have 
declined over this period. It may be that some of the ‘novelty effect’ of 
reading, and of being regularly tested on their comprehension of this reading, 
had worn off by the later part of the school year. 

Survey findings: treatment and comparison schools 

The information collected from the surveys of pupils also enabled the research 
team to compare pupils’ attitudes, confidence and motivation across program 
and non-program schools. In other words, the sweep 2 survey responses help 
us to answer the question: ‘Were pupils in the Renaissance schools better 
motivated towards mathematics and reading than those in schools where the 
programs were not being used?’ 

Mathematics in primary schools 

On the whole, as at June 2006, Year 4 and 5 pupils in the program schools had 
more positive views of mathematics than pupils in comparable schools where 
the program was not being used. 

• Whilst pupils using Accelerated Maths were slightly less confident about 
doing well in mathematics (44 per cent agreed ‘a lot’ that that they usually 
did well, compared to 52 per cent in comparison schools), for most other 
attitude items, a considerably greater proportion of pupils in treatment 
schools agreed ‘a lot’ with a positive statement than in comparison 
schools. For example: 

y 54 per cent of Renaissance pupils would like to do more mathematics 
at school, compared to 39 per cent in non-program schools 

y 65 per cent of Renaissance pupils enjoyed learning mathematics, 
compared with 56 per cent of pupils in non-program schools 

y 62 per cent of Renaissance pupils said that the work that they did in 
mathematics was interesting, compared to 49 per cent in non-program 
schools. 

• Over half of pupils using the Renaissance program found mathematics 
‘very enjoyable’ compared with 45 per cent of non-program pupils. 

• However, a larger proportion of comparison pupils felt that that they had 
made ‘a great deal of progress’ in mathematics (52 per cent of pupils in 
comparison schools and 46 per cent in treatment schools, though the 
figures are very evenly matched if the pupils indicating ‘some progress’ 
are also included (40 and 43 per cent respectively). 
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Mathematics in secondary schools 

• Year 7 pupils in program schools had a lower perception with regard to 
‘doing well’ in mathematics (35 per cent agreed ‘a lot’ that they ‘usually 
do well in maths’) than the same age group in comparison schools (where 
64 per cent agreed ‘a lot’). 

• In terms of perceived enjoyment of mathematics lessons, there was 
something of a polarisation of program pupils, with higher proportions 
finding mathematics ‘very enjoyable’ (26 per cent compared to 20 per 
cent) and ‘not very enjoyable’ (23 per cent compared to 14 per cent) 
compared with non-program pupils. 

• The survey also revealed that pupils in program schools had a much lower 
likelihood of expressing a view that they had made ‘a great deal of 
progress’ in mathematics (39 per cent compared to 64 per cent). It should 
be noted, however, that all survey respondents in all schools reported 
making at least ‘a little’ progress. 

Reading in primary schools 

There is some evidence to suggest that Year 4/5 pupils in Accelerated Reader 
schools had more positive attitudes to reading than their peers in non-program 
schools. For example: 

• were more likely to say that they liked ‘reading stories’ (73 per cent 
compared to 63 per cent), and that they enjoyed reading (69 per cent 
compared with 53 per cent). 

• Year 4/5 pupils in Accelerated Reader schools were more likely to read 
every day than their peers in non-program schools, though the figures are 
very similar when pupils who read ‘most days’ are also considered. 

• Likewise, although more pupils in program schools found reading lessons 
‘very enjoyable’, the figures for treatment and comparison schools are 
very similar when the ‘very’ and ‘quite’ enjoyable categories are 
combined. 

• Pupils in Accelerated Reader primary schools were slightly more likely to 
report ‘a great deal of progress’ in reading than those in non-program 
schools (53 per cent compared to 48 per cent). 

Reading in secondary schools 

There is also evidence that Year 7 pupils in Accelerated Reader schools had 
more positive attitudes to reading than their peers in non-program schools. For 
example: 

• Pupils in program schools were more likely to say that they liked ‘reading 
stories’ (72 per cent compared to 46 per cent), and that they enjoyed 
reading (64 per cent compared with 46 per cent) (Table 26). 

• As was the case with primary school participants, Year 7 pupils in 
Accelerated Reader schools were considerably more likely to read every 
day than Year 7 pupils in non-program schools (19 per cent compared to 
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three per cent), though the figures are very similar when pupils who read 
‘most days’ are also considered. 

• Year 7 pupils using the Renaissance program were much more likely to 
enjoy reading (79 per cent indicating that reading was ‘very’ or ‘quite’ 
enjoyable, compared with 56 per cent in non-program schools), and the 
former group were also more likely to express the opinion that they had 
made ‘a great deal of progress’ with their reading (38 per cent compared 
with 28 per cent). 

Interview data: teacher perspectives 

In order to collect complementary qualitative data, the research team visited 
four case-study schools (primary mathematics, primary reading, secondary 
mathematics, secondary reading) and interviewed two teachers involved with 
the programs in each school (in one school, only one teacher was involved 
with using the program, so a total of seven teachers were interviewed). 

In all the schools the staff interviewed had a positive attitude about using the 
Renaissance Learning programs. In the primary school using the mathematics 
program, there was seen to be a particular advantage in ‘the fact that the 
computer does the marking, which frees teachers to deal with pupils who are 
having difficulties’. The point about the appeal of using computers and the 
‘instant feedback’ they gave was repeated in all the schools and was reflected 
in this view: ‘The students l ik e it because they get feedback, they get reports 
and it makes them independent’. 

The two schools using the mathematics program agreed that it was a useful 
addition to the strategies they already used, for, as one teacher explained: ‘It 
does not replace the teaching, but enables teachers to pinpoint who needs help 
and on what topic’. The secondary reading school was particularly positive 
about the program’s contribution to personalised learning and its effectiveness 
in helping a situation where there was a huge variation in reading ages at the 
beginning of Year 7. Pupils could work on the program in their own time and 
many did, because ‘it has motivation built into it with the points system’ and 
this was reinforced with their own school awards. 

In two of the schools (one primary mathematics and one secondary reading), 
staff felt that using the Renaissance Learning program had strengthened links 
with parents. In the primary school, the program had helped to involve parents 
more closely with their children’s work: 

The marked tests go home and so parents get to see how their children 
are doing and they are given directions to help the children with their 
work. Parents do come in and talk about their children 's progress and 
what they are doing. 

In one of the secondary schools, the use of the reading program was reported 
as having made an important contribution to the school’s family learning 
programme: ‘the parents now understand why their children bring books home 
and they can support them’. 
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Staff in all four schools said that they had been faced with technological 
problems when they first started using the Renaissance Learning programs. 
This was the only type of negative comment made about using the programs. 
Issues such as ‘constant crashing’, ‘the printer not working’, an unreliable pass 
word system and ‘some answers were correct, but the software flagged them 
up as wrong’, had caused irritation. This situation had improved considerably 
after a time, so that all the schools agreed that: ‘It is better now as I have a 
telephone link with Renaissance Learning. They are helpful in that way’. 

The extent of the staff training necessary to run the programs was referred to 
by all the school interviewees, as was the need for a ‘fair bit of commitment 
from one or two teachers to start with, in order to get really familiar with it’. If 
a school had high staff turnover, as one in particular did, this could cause 
problems. 

All the interviewees agreed that one of the most significant strengths of the 
Renaissance Learning programs was that they motivated pupils. Pupils on both 
the Mathematics and Reading programs responded well to being in charge of 
their own rate of progress and seeing that progress confirmed. From the 
teachers’ point of view too, one of the main advantages was the personalised 
learning aspect of the programs, which in the words of one interviewee, 
‘addresses the difficult issue of differentiating teaching and learning in very 
widely mixed ability classes’. 

From a practical point of view, the way the programs worked was seen in all 
the schools as aiding monitoring and target setting and therefore also 
reporting. In the primary mathematics school, it was pointed out that they ‘can 
print out the diagnostic tests so the teacher can see very clearly in which areas 
they are not getting on and need extra support’. It also meant that ‘the marked 
tests and scores go home with the children at the end of every week’. 
Similarly, in the secondary reading school, the interviewee explained that the 
program helped the school to ‘use support and intervention more effectively’, 
and that ‘the monitoring and intervention are real strengths’. 

As a final question, the interviewees were asked if they would recommend the 
use of the Renaissance Learning program to colleagues in their own and other 
schools and all said that they would. 
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1. Introduction 



1.1 Background 

Renaissance Learning programs are designed to help improve pupils’ reading, 
mathematics and writing skills. With the aid of computer software, teachers 
are able to devise personalised learning programmes for pupils that allow them 
to develop at a pace suitable for the individual. The programs are also thought 
to be helpful for professional development in that they can improve teachers’ 
instructional practices and assist with assessment and diagnosis. 

The Renaissance programs have been widely used in the United States and, in 
the academic year 2004-5, the reading and mathematics programs were piloted 
in a small number of Specialist Schools (and their feeder schools) in London. 
In the pilot year, the Reading programme was implemented in three Specialist 
Schools (and seven feeder schools) and the mathematics programme was 
implemented in two Specialist Schools (and four feeder schools). The phased 
introduction of these programmes continued in 2005-6, in both Specialist 
secondary schools and primary feeder schools, and was overseen by the 
Specialist School and Academies Trust’s Regional Coordinator for London 
and the South-East. 

It was in this context that Renaissance Learning commissioned the National 
Foundation for Educational Research (NFER) to carry out a rigorous, 
independent evaluation of the implementation of these programs in 2005-6. 
This report presents the findings from this evaluation, including: 

• analysis of pupils’ progress over time (over the academic year 2005-6) 
based on the use of ‘before and after’ reading and mathematic tests in the 
schools using the programs 

• comparative analysis of pupil progress and attitudes in: (i) schools using 
the Renaissance programs (‘treatment’ schools); and (ii) schools not using 
these programs (‘comparison’ schools). 



Renaissance Learning recommends a set of teaching techniques that use the 
Accelerated Reader program ( www.renlearn.com) . The Accelerated Reader is 
based on the principal that practice improves reading skills. Pupils read books 
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based on their instructional reading level, and then complete an Accelerated 
Reader quiz on the computer that matches the book. All the books used are in 
line with the requirements of the national curriculum for English at the 
relevant key stages. The Accelerated Reader provides teachers with 
information to be able to monitor and guide each pupil’s reading practice and 
develop a learning program suitable to the personal needs of the pupil. This 
has been shown to improve students’ reading achievement and reading 
attitudes. 1 

Accelerated Maths follows a similar strategy, using a combination of 
computer software and teaching methods recommended by Renaissance 
Learning to assess pupils’ skills and competencies. The strategy is thought to 
help teachers develop assignments for students that are pitched at the 
appropriate level and that allow pupils to develop at their own pace. In this 
respect the strategies fit with current moves towards ‘personalised’ and 
‘blended’ learning, and the increasing use of e-assessment. 

Some initial evaluation work was completed between November 2004 and 
January 2005. This involved participating schools completing a round of tests 
and student questionnaires. These provided useful information, but the 
evaluation was not progressed into the second half of the academic year 
because it became apparent that, in the pilot stages, there was much variation 
in implementation, depending upon curriculum requirements, staff availability 
and logistical circumstances (for example, some schools were awaiting the 
installation of broadband technology). For these reasons, Renaissance 
Learning and the NFER agreed that the full one-year evaluation should not 
take place until the second year of implementation of the Renaissance 
Programs. The main aims of the evaluation are outlined in the next section. 



1.2 Aims 



The central aim of the study was: to evaluate the effectiveness of Accelerated 
Reader and Accelerated Maths in selected Specialist and feeder schools in 
London. ‘Effectiveness’ is defined, based on an initial research specification 
put together by Renaissance Learning, as: ‘accelerating students’ acquisition 



1 Holmes, T. and Brown, C. (2003). A Controlled Evaluation of a Total School Improvement 

Process, School Renaissance. Paper presented at the National Renaissance Conference, Nashville, 
TN, February 7, 2003. 
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of reading and maths skills, improving their motivation to read and perform 
maths tasks, and improving the teachers’ instructional practices’. 

Relating to this broad aim, there were a number of more specific research 
questions. The evaluation sought to: 

• examine the differences in performance in standardised reading and 
mathematics tests between pupils participating in Renaissance Programs 
and those who were not 

• assess, as far as possible, within the methodological approaches selected, 
the extent to which any gains in pupil performance could be attributed to 
Renaissance program implementation 

• to examine pupil perceptions and experiences of the Renaissance 
Programs, along with their views on how these programs compare with 
other forms of learning reading and mathematics. 



The next section of this report sets out details of the methodological 
approaches that were used to address these project aims. 



1.3 Methodology 

The research was conducted within a quasi-experimental framework, as 
required by the research specification. This involved matching Specialist and 
feeder schools implementing the programs (‘treatment’ schools) with similar 
schools where the program was not being implemented (referred to as 
‘comparison’ schools). 

Samples of schools, classrooms and students 

In all, 21 schools assisted with the research: 14 of these were schools using the 
Renaissance Programs and seven were schools not using the programs, but 
which were similar to the treatment schools in other respects. All of the 
schools using the programs were using either the Mathematics programme or 
the Reading program. Further details of the sample schools are provided in 
Table 1.1 below. 
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Table 1.1 Treatment and comparison school samples 



Treatment Schools 


Comparison Schools 


Using a Renaissance Program 


Not using a Renaissance Program 


Mathematics 2 Specialist 


4 feeder 


1 Specialist 


3 feeder 


Schools 


schools 


School 


schools 


112 pupils 


220 pupils 


37 pupils 


133 pupils 


Reading 2 Specialist 


6 feeder 


1 Specialist 


2 feeder 


Schools 


schools 


School 


schools 


101 pupils 


225 pupils 


66 pupils 


41 pupils 




Total number of schools in sample 






21 


Total number of pupils in sample (based 


on Sweep 1 test returns) 


935 



The sample of treatment schools was determined by school participation in 
the Renaissance programs, based upon information supplied by Renaissance 
Learning. The selection of comparison schools was based upon details of 
school characteristics obtained from the NFER’s National Register of Schools 
Database. This enabled the research team to match schools with broadly 
similar characteristics, including proportions of pupils eligible for free school 
meals (FSM), which can be seen as a proxy indicator of the socio-economic 
circumstance of a school. 

Information provided by Renaissance Learning indicated that the Renaissance 
programmes were, in most cases, being delivered to at least two classes of 
pupils in each school. The target classroom sample therefore consisted of two 
participating classes from each treatment and comparison school. In some 
cases, however, the program was being piloted with just one class in the year 
group. 

Within this sampling framework, the evaluation had two main methodological 
elements: (1) standardised testing; and (2) pupil surveys. Details of the tests 
used are provided in Chapter 2 and the pupil surveys are included as Appendix 
2. The tests and surveys were completed twice by the same groups of pupils, 
firstly in November 2005 and secondly, towards the end of the school year, in 
June 2006. Face-to-face interviews were also conducted with teachers in four 
of the schools using the Renaissance programs (see below). 
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Student questionnaire surveys 

In addition to obtaining student attainment data, the research team also made 
an assessment of pupil experiences of using the Renaissance programs, or 
other forms of mathematics/literacy learning, both in the treatment and the 
comparison schools. This was done through the use of pupil attitudinal 
questionnaire surveys which were conducted, in Years 4/5 and 7, at two points 
in time. Most of the individual questions used in these surveys had featured in 
previous NFER evaluations relating to numeracy and literacy so, to an extent, 
their reliability had already been established. 

This approach enabled comparisons to be made of pupils’ attitudes and 
perceptions both across time and across program and non-program schools. 
Since the survey sample was the same as the test sample, it was possible to 
examine the survey findings alongside the performance data for the pupils. 
The scale of the project did not allow for the use of individual matched pupil 
data, but the survey and test approach meant that attitudes and motivational 
factors could be captured, as well as any changes in attainment in reading and 
mathematics. 

Teacher interviews 

The evaluation also included consideration of teacher satisfaction and 
experiences with the programs. To obtain this information the research team 
conducted semi- structured interviews with a sample of teachers in the 
treatment schools. The rationale behind this was as follows: 

• the numbers of teachers involved (even if comparison schools were 
included) would have been relatively small - certainly not large enough 
for a questionnaire survey to produce meaningful statistical analyses 

• teachers in the comparison schools, whilst being able to comment on their 
mathematics and literacy teaching, would not have been able to have said 
anything about the Renaissance programs 

• teachers in the treatment schools were able to comment on both their 
experiences of the Renaissance programs, and their experiences of other 
frameworks/schemes for reading and mathematics (which they would have 
used prior to the implementation of the Renaissance programs). 
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The aim was to interview two teachers involved in implementing the programs 
in each of four schools, giving a total of eight teacher interviewees . 2 The 
interview data provided useful additional qualitative/perceptual information to 
complement the quantitative data collected via the tests and pupil 
questionnaire surveys. 



1.4 Report structure 

The following chapter examines trends in attainment, in both primary and 
secondary schools, based upon the two sets of test returns, over the duration of 
the evaluation. In the next part of the report, Chapters 3 and 4 present the 
findings from the pupil questionnaire survey, reporting on attitudinal and 
motivational changes relating to mathematics and reading, respectively. 
Chapter 5 presents the findings from the interviews with teachers. Finally, 
Chapter 6 pulls the data together and summarises the main findings from the 
evaluation, including the key issues and benefits arising from use of the 
programs. This chapter also offers a number of recommendations relating to 
the possible future development of the programs. Copies of the surveys and 
interview schedule used are included as appendices to the report. 



In practice, only one teacher was available in one of the schools, so a total of seven teachers were 
interviewed. 
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2. Pupil attainment 



2.1 The standardised tests 



In order to make an assessment of progression in pupil performance, 
standardised tests were carried out at two points in the school year, one early 
on (by November 2005, sweep 1) and another later on in the year (June 2006, 
sweep 2). These were carried out with Year 4 or 5 (depending on the target 
group in the primary schools and Year 7 pupils in both ‘treatment’ and 
comparison schools. The pre-test post-test approach allowed comparisons over 
time (approximately an eight-month period) and across program and non- 
program schools. 



It should be stressed that, where there are any differences in pupil performance 
over time or across participating and non-participating schools, it may not be 
possible to attribute causality to the Renaissance Programs. The reasons for 
this include the following: 

• Firstly, although the sample sizes as a whole are relatively large - 
sufficient test papers for analysis were returned from 21 schools and 935 
pupils were tested in sweep 1 and 894 in sweep 2 - the sub-samples are 
smaller, especially for the comparison groups, and caution needs to be 
exercised when these are used. 

• Secondly, the time period for the evaluation, consisting of approximately 
eight months between the tests, was relatively short. 3 This means that any 
improvements in attainment are likely to be limited, because there has 
only been a relatively short period of time for the programs to have an 
impact. In addition, the research literature in the areas of pupil self- 
efficacy and motivation show that it can take some considerable time for 
improvements in these areas to become manifest. 4 

• Thirdly, the quasi-experimental approach applied to schools cannot 
overcome the effects of multiple variables. In other words, although the 
use of comparison schools enabled a degree of ‘control’ of certain 
influential factors, there were many others factors that could have affected 
pupil attainment other than the use (or non-use) of the Renaissance 
Programs. These could have included the effects of individual teachers, 



3 In addition there were school holidays in this period, so the amount of time that teachers and pupils 
actually had to implement the Renaissance Programs was somewhat less than six months. 

4 See, for example: Husman, J., Brem, S. and Duggan, M.A. (2005). ‘Student goal orientation and 
formative assessment'. Academic Exchange Quarterly, 9, 3, 355-359. 
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changes in staff, and the impact of other initiatives to do with reading and 
mathematics. 

• Finally, it is worth emphasising that it was not possible for this research to 
take full account of variations in the levels of implementation of the 
Renaissance programs. The implementation and monitoring of the 
programs are very much teacher-centred, meaning that the teacher has the 
defining influence in how the programs are used in his/her classroom. 
Implementation levels vary and therefore pupils in treatment schools will 
have had different levels of experience of the Renaissance programs. 



The point about varying implementation levels has been emphasised in 
previous studies of the use of the Renaissance programs. Over the past twenty 
years Renaissance Learning has identified the “best classroom practices” for 
these programs and has used these as a basis for making recommendations 
about the use of the programs . 5 The training of teachers is based upon these 
practices and recommendations and has been developed and modified on the 
basis of analyses of pupil performance data and a number of evaluations. The 
findings from previous evaluations provide evidence that, when schools 
implement the Accelerated Reader or Accelerated Mathematics programs 
according to these recommendations, they tend to experience better than 
average pupil attainment gains. Topping, for example, in a study of the use of 
the Accelerated Reader program in a very similar sample of schools to the one 
used here, concluded that the impact of the program was ‘remarkably 
successful’ and, furthermore, ‘had the schools implemented Accelerated 
Reader stringently, the outcomes would have been even better ’. 6 



Despite these qualifying comments, it needs to be stressed that the use of a 
combination of test data, survey data, and teacher perspectives has made the 
evaluation stronger and allowed for the triangulation of findings and data 
sources. The test data can be seen as being indicative of the general progress 
made (or not made) by the pupils and provide a useful overall picture of 
attainment trends for this school year. In addition, the use of standardised tests 
means that any improvements in attainment identified have occurred over and 
above what would be expected given the expected age-related progress of the 
pupil samples. 



5 Details of Renaissance Learning's best practice recommendations and figures on implementation 
levels for individual schools are presented in Tables 2.2 and 2.5. 

6 Topping, K. (2006). Accelerated Reader in Specialist Schools. Unpublished report. 
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The tests were ‘off the shelf tests which are widely used for measuring the 
reading/mathematics skills of pupils in these year groups. The tests were 
purchased from NFER-Nelson and were teacher administered, but NFER- 
Nelson and the NFER handled the scoring of the assessments, using 
experienced markers. The mathematics tests were from the new NFER-Nelson 
Progress in Mathematics 4-14 series. These were different tests for each age 
group (Years 4/5 and 7) and for each time of the year. A sentence completion 
test, the Suffolk Reading Scale, Second Edition, was used as the reading test. 
This is a comprehensive and flexible measure of pupils’ reading ability and is 
one of the most commonly-used reading tests in the UK. 



2.2 Pupil attainment in mathematics 



Table 2.1 below summarises trends in pupil attainment in mathematics, based 
on standardised test scores, on a school-by-school basis. Schools A to F 7 are 
the treatment schools, where the Accelerated Maths program has been in use. 
Schools G-J are the non-program, comparison schools. The main findings 
from this table can be summarised as follows: 

• primary schools: average standardised test scores for mathematics 

improved in two out of four treatment schools; in the three comparison 
schools, one saw a decline in average score and two saw improvements 

• secondary schools: two treatment secondary schools returned their tests, 
in both cases the average standardised score improved from sweep 1 to 
sweep 2. The improvements were considerable, over eight points in each 
case. One comparison secondary school returned their mathematics tests: 
the average standardised score in this school also improved, but not to the 
extent that was evident in the treatment schools. 

As noted in the previous section, it is important to bear in mind that the degree 
to which Accelerated Maths has been implemented in the treatment schools 
will vary. Information about the quantity and quality of implementation is 
collected by Renaissance Learning, and indicators of these are provided in 
Table 2.2, which needs to be considered alongside Table 2.1. Current best 
practice implementation recommendations for Accelerated Maths are that: (i) 
pupils should work to master two to four objectives per week, depending on 
their year group; and (ii) they should average above 85 per cent on their tests. 



7 Schools agreed to complete the tests and the surveys on the basis of confidentiality and anonymity. 
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Table 2.1 Attainment in mathematics: individual schools 



Mathematics 




Sweep 


Year 


Pupils 


Total 


Average 


Change 


School A 




Sweep 1 


Yr 7 


55 


4706 


85.56 




Maths Treatment 




Sweep 2 


Yr 7 


58 


5459 


94.12 


8.56 


School B 




Sweep 1 


Yr 7 


57 


5542 


97.23 




Maths Treatment 




Sweep 2 


Yr 7 


28 


2951 


105.39 


8.16 


School C 




Sweep 1 


Yr 4 


58 


5674 


97.83 




Maths Treatment 




Sweep 2 


Yr 4 


58 


5703 


98.33 


0.50 


School D 




Sweep 1 


Yr 4 


52 


4882 


93.88 




Maths Treatment 




Sweep 2 


Yr 4 


39 


3561 


91.33 


-2.55 


School E 




Sweep 1 


Yr 5 


58 


5666 


97.69 




Maths Treatment 




Sweep 2 


Yr 5 


56 


5455 


97.41 


-0.28 


School F 




Sweep 1 


Yr 5 


52 


4939 


94.98 




Maths Treatment 




Sweep 2 


Yr 5 


56 


5667 


101.20 


6.22 


School G 




Sweep 1 


Yr 7 


37 


3836 


98.36 




Maths Comparison 




Sweep 2 


Yr 7 


44 


4525 


102.84 


4.48 


School H 




Sweep 1 


Yr 4 


26 


2579 


99.19 




Maths Comparison 




Sweep 2 


Yr 4 


24 


2000 


83.33 


-15.86 


School 1 




Sweep 1 


Yr 5 


54 


5428 


100.52 




Maths Comparison 




Sweep 2 


Yr 5 


58 


6077 


104.78 


4.26 


School J 




Sweep 1 


Yr 5 


53 


5225 


98.58 




Maths Comparison 




Sweep 2 


Yr 5 


57 


5668 


99.44 


0.86 


Table 2.2 Implementation data for Accelerated Maths: individual schools 


Mathematics 


Year 


No. of 
pupils 8 


Implementation 

(Quantity) 


No. of Implementation 

pupils (Quality) 


School A 


Yr 7 


65 




34.9 


149 




82% 


School B 


Yr 7 


60 




4.6 


60 




77 % 


School C 


Yr 4 


125 




24.1 


241 




85 % 


School D 


Yr 4 


85 




0 


85 




— 


School E 


Yr 5 


60 




39.6 


60 




86% 


School F 


Yr 5 


58 




2.1 


58 




84% 



Implementation quantity: this figure is the average number of objectives achieved by pupils 
Implementation quality: this figure is the average percentage of correct answers for total tests 
Source: Implementation data provided by Renaissance Learning 



Schools A-F correspond with schools A-F in Table 2.1. The number of pupils varies because 
Renaissance Learning collected information for all pupils involved, whereas the NFER tests were 
taken by one or two classes of pupils in each school. 
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Table 2.3 below provides an aggregated analysis of mathematics test scores 
for treatment and comparison schools. It can be seen that, taking all the 
schools using Accelerated Maths, the average standardised pupil score (N=332 
in sweep 1) increased by three points from 94.61 to 97.61; in the comparison 
schools (not using the program) the average standardised pupil score (N=170) 
decreased from 100.40 to 99.84. 



Table 2.3 Attainment in mathematics: treatment and comparison schools 



Accelerated Maths 


Sweep 1 


Sweep 2 




Pupils 


Average 


Pupils 


Average 


Treatment primary schools - maths 


220 


96.19 


209 


97.54 


Treatment secondary schools - maths 


112 


91.50 


86 


97.79 


Comparison primary schools - maths 


133 


99.49 


139 


98.88 


Comparison secondary schools - maths 


37 


103.68 


44 


102.84 


All treatment schools - maths 


332 


94.61 


295 


97.61 


All comparison schools - maths 


170 


100.40 


183 


99.84 



2.3 Pupil attainment in reading 



Table 2.4 below summarises trends in pupil attainment in reading, based on 
standardised test scores, on a school-by-school basis. Schools K to R are the 
treatment schools, where the Accelerated Reading program has been in use. 
Schools S-U are the non-program, comparison schools. The main findings 
from this table can be summarised as follows: 

• primary schools: average standardised test scores for reading improved 
in five out of six treatment schools; in the two comparison schools, one 
saw a decline in average score and one saw an improvement 

• secondary schools: two treatment secondary schools returned their tests, 
in both cases the average standardised score declined from sweep 1 to 
sweep 2. One comparison secondary school returned their reading tests: 
the average standardised score in this school also declined. 



As with the mathematics scores, account needs to be taken of the reported 
implementation levels of the Accelerated Reader in these schools. 
Information about the quantity and quality of implementation is collected by 
Renaissance Learning, and indicators of these are provided in Table 2.5, which 
needs to be considered alongside Table 2.4. Current best practice 
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implementation recommendations for Accelerated Reader are that: (i) pupils 
should average above 85 per cent on the book quizzes (above 90 per cent for 
secondary pupils), and (ii) they should read for 30 minutes a day (20 minutes 
for secondary). 



Table 2.4 Attainment in reading: individual schools 



Reading 


Sweep 


Year 


Pupils 


Total 


Average 


Change 


School K 


Sweep 1 


Yr 7 


57 


5659 


99.28 




Reading Treatment 


Sweep 2 


Yr 7 


46 


4521 


98.28 


-1.00 


School L 


Sweep 1 


Yr 7 


44 


3896 


88.55 




Reading Treatment 


Sweep 2 


Yr 7 


44 


3738 


84.95 


-3.60 


School M 


Sweep 1 


Yr 4 


12 


1071 


89.25 




Reading Treatment 


Sweep 2 


Yr 4 


12 


1088 


90.67 


1.42 


School N 


Sweep 1 


Yr 4 


28 


2691 


96.11 




Reading Treatment 


Sweep 2 


Yr 4 


25 


2421 


96.84 


0.73 


School 0 


Sweep 1 


Yr 4 


50 


4587 


91.74 




Reading Treatment 


Sweep 2 


Yr 4 


46 


4239 


92.15 


0.41 


School P 


Sweep 1 


Yr 4 


47 


4093 


87.09 




Reading Treatment 


Sweep 2 


Yr 4 


46 


4057 


88.20 


1.11 


School Q 


Sweep 1 


Yr 4 


55 


5035 


91.55 




Reading Treatment 


Sweep 2 


Yr 4 


51 


4950 


97.06 


5.51 


School R 


Sweep 1 


Yr 4 


33 


3275 


99.24 




Reading Treatment 


Sweep 2 


Yr 4 


42 


3811 


90.74 


-8.50 


School S 


Sweep 1 


Yr 7 


41 


3948 


96.29 




Reading Comparison 


Sweep 2 


Yr 7 


39 


3664 


93.95 


-2.34 


School T 


Sweep 1 


Yr 4 


30 


2809 


93.63 




Reading Comparison 


Sweep 2 


Yr 4 


25 


2486 


99.44 


5.81 


School U 


Sweep 1 


Yr 4 


36 


3434 


95.39 




Reading Comparison 


Sweep 2 


Yr 4 


40 


3810 


95.25 


-0.14 
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Table 2.5 Implementation data for Accelerated Reader: individual schools 



Reading 


Year 


No. of pupils 

9 


Implementation 

(Quantity) 


Implementation 

(Quality) 


School K 


Yr 7 


130 


3.2 


67 % 


School L 


Yr 7 


235 


9.5 


66% 


School M 


Yr 4 


25 


13.3 


83 % 


School N 


Yr 4 


66 


40.9 


80% 


School O 


Yr 4 


52 


16.4 


83 % 


School P 


Yr 4 


133 


3.0 


71 % 


School Q 


Yr 4 


55 


9.1 


79 % 


School R 


Yr 4 


109 


3.2 


64 % 



Implementation quantity: this figure is the average of pupil estimated minutes spent reading 
Implementation quality: this figure is the average of pupil average percentage correct answers 
Source: Implementation data provided by Renaissance Learning 



Table 2.6 below provides an aggregated analysis of reading test scores for 
treatment and comparison schools. It can be seen that, taking all the primary 
schools using the Accelerated Reader program, the average standardised pupil 
score (N=225 in sweep 1) increased marginally from 92.23 to 92.64; in the 
comparison primary schools (not using the program), however, the average 
standardised pupil score (N=66) also increased, and by a bigger margin. 

Table 2.6 Attainment in reading: treatment and comparison schools 



Accelerated Reader Sweep 1 Sweep 2 





Pupils 


Average 


Pupils 


Average 


Treatment primary schools - reading 


225 


92.23 


222 


92.64 


Treatment secondary schools - reading 


101 


94.60 


90 


91.77 


Comparison primary schools - reading 


66 


94.59 


65 


96.86 


Comparison secondary schools - reading 


41 


96.29 


39 


93.95 


All treatment schools - reading 


326 


92.97 


312 


92.39 


All comparison schools - reading 


107 


95.24 


104 


95.77 



2.4 Overall attainment 



Taking all the schools returning tests for this evaluation, in both mathematics 
and reading, it was found that in the 14 schools using Renaissance Programs 



9 Schools K-R correspond with schools K-R in Table 2.4. The number of pupils varies because 
Renaissance Learning collected information for all pupils involved, whereas the NFER tests were 
taken by one or two classes of pupils in each school. 
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the average standardised pupil score (N=658 in sweep 1) increased by nearly 
half a point from 93.79 to 94.23, whereas in the seven comparison schools the 
average standardised pupil score (N=277) decreased very slightly, from 98.41 
to 98.36 (Table 2.7). 



Table 2.7 Attainment in mathematics and reading: 
treatment and comparison schools 



Accelerated Maths 
& Accelerated Reader 


Sweep 1 

Pupils Average 


Sweep 2 

Pupils Average 


All treatment primary schools 


445 


94.19 


431 


95.02 


All treatment secondary schools 


213 


92.97 


176 


94.71 


All comparison primary schools 


199 


97.86 


204 


98.24 


All comparison secondary schools 


78 


99.79 


83 


98.66 


All treatment schools 


658 


93.79 


607 


94.23 


All comparison schools 


277 


98.41 


287 


98.36 



Average standardised score for pupil 






using Renaissance 


up from: 


93.79 to 94.23 (+0.46) 


Average standardised score for pupil not 


down 




using Renaissance 


from: 


98.41 to 98.36 (-0.05) j 



Summary 

The key overall finding regarding pupil attainment was that the most 
significant improvements occurred in primary reading and secondary 
mathematics schools. In primary mathematics and secondary reading schools, 
the findings are less conclusive though, as noted in Section 2.1 above, there 
are many reasons, unconnected with the use or non-use of the Renaissance 
programs, as to why this might be so. In addition to these broad trends, there 
are some individual schools which achieved notable successes in a short space 
of time. 
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3. Pupil attitudes: mathematics 



The Renaissance Programs, through the use of regular built-in assessment and 
testing, enable pupils and teachers to see what progress is being made in 
attainment. The programs are also designed to meet individual learning 
requirements and to encourage and motivate pupils in reading and 
mathematics. For this reason, questionnaire surveys were issued to the same 
groups of pupils taking the standardised tests. The information collected from 
the surveys of pupils enabled the research team to identify any changes in 
attitudes, confidence or motivation in pupils in treatment schools over the 
academic year (the same questionnaire was completed twice), and also to 
compare the attitudes of pupils in program schools with those of pupils in non- 
program schools. 

This chapter examines the findings from the pupil attitude questionnaire 
relating to mathematics. Sections 3.1 and 3.2 look at changes in pupil 
attitudes over time by comparing sweep 1 and sweep 2 findings from all the 
schools, dealing with primary schools and secondary schools respectively. 
Sections 3.3 and 3.4 compare attitudes to mathematics in program and non- 
program schools. Chapter 4 takes a similar approach in relation to the findings 
concerning attitudes towards reading. 10 



3.1 Mathematics: progress over time - primary schools 

Table 3.1 below shows that, overall, there was little change over time with 
respect to Year 4/5 pupils’ views of how well they were doing in mathematics, 
although there were slightly more pupils disagreeing that they were not good 
at mathematics (51 per cent in sweep 1 increased to 55 per cent in sweep 2). 



10 p or reasons 0 f space and clarity. Chapters 3 and 4 report only the findings from selected 
questionnaire items. In the main, these are items where some change in attitudes has occurred, or 
where there was a noticeable difference in attitudes across program and non-program schools. 
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Table 3.1 



Years 4 and 5: Levels of agreement with statements about learning 
maths 



Views about Agree a Agree a Disagree a Disagree a No 

Mathematics lot little little lot response 



% % % % % 





SI 


S2 


SI 


S2 


SI 


S2 


SI 


S2 


SI 


S2 


I usually do well in 
maths 


46 


47 


43 


42 


5 


6 


3 


2 


2 


3 


I would like to do 
more maths at school 
Maths is harder for me 


56 


48 


22 


26 


11 


14 


5 


6 


5 


5 


than for many of my 
classmates 


12 


9 


21 


20 


30 


28 


33 


37 


4 


6 


I enjoy learning maths 


68 


63 


20 


22 


4 


6 


3 


3 


5 


6 


I am just not good at 
maths 


8 


6 


18 


14 


17 


18 


51 


55 


5 


7 


I learn things quickly 
in maths 


38 


39 


37 


38 


15 


12 


6 


4 


3 


7 


The work I do in 
maths lessons is 
interesting 


65 


58 


21 


28 


6 


5 


4 


4 


3 


7 


N = 


330 


330 



















SI and S2 refer to Sweep 1 and Sweep 2 
A series of single response items 

Due to rounding, percentages may not always sum to 100 
Source: NFER Year 4/5 Mathematics Survey 2005/6 

Year 4/5 respondents expressed mixed feelings about their levels of enjoyment 
of recent mathematics lessons (see Table 3.2 below), with fewer pupils finding 
them ‘very’ enjoyable by the end of the year (58 per cent decreased to 51 per 
cent), but more finding them ‘quite’ enjoyable (32 per cent increased to 38 per 
cent). 



Table 3.2 Years 4 and 5: Level of enjoyment of most recent maths lessons 



Level of enjoyment of most recent maths lesson 


Sweep 1 

% 


Sweep 2 

% 


Very enjoyable 


58 


51 


Quite enjoyable 


32 


38 


Not very enjoyable 


5 


6 


Not at all enjoyable 


2 


2 


No response 


2 


3 


N = 


330 


330 



A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 4/5 Mathematics Survey 2005/6 
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There was, however, a positive development in attitudes to progress in 
mathematics (Table 3.3), with more Year 4/5 respondents believing that they 
had made a ‘great deal’ of progress by sweep 2 (41 per cent increased to 48 
per cent) and a slight fall in the percentage believing they had made ‘no 
progress’. 



Table 3.3 Years 4 and 5: Perceived progress in maths over last six months 



Progress made in maths over the last six months 


Sweep 1 

% 


Sweep 2 

% 


A great deal of progress 


41 


48 


Some progress 


48 


43 


A little progress 


9 


5 


No progress 


2 


1 


No response 


1 


3 


N = 


330 


330 



A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 4/5 Mathematics Survey 2005/6 



3.2 Mathematics: progress over time - secondary schools 

Secondary school pupils were asked to complete questionnaires on their 
attitudes to mathematics at the same time as the primary schools, but it should 
be noted that, partly because there were fewer secondary schools using 
Accelerated Maths, the response rate of those who completed both sweep 1 
and sweep 2 was much lower, with a total of 38 pupils returning 
questionnaires. The responses across time, to questions about learning 
mathematics, were broadly positive (Table 3.4), with a particular increase in 
pupils who agreed ‘a lot’ that they did well in mathematics (for 50 to 61 per 
cent). 
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Table 3.4 Year 7: How much do you agree with these statements about 
learning maths? 



Views about mathematics 


Agree a 


Agree a 


Disagree a 


Disagree 






lot 






little 


little 




a lot 






% 






% 


% 




% 






SI 




S2 


SI 


S2 


SI 


S2 


SI 


S2 


1 usually do well in maths 


50 




61 


45 


37 


5 


3 


0 


0 


1 would like to spend more 
time studying maths at 
school 


16 




11 


39 


58 


34 


21 


11 


11 


Maths is more difficult for 
me than for many of my 
classmates 






8 


18 


18 


26 


34 


55 


39 


1 enjoy learning maths 


47 




34 


37 


50 


13 


11 


3 


5 


Sometimes, when I do not 
understand a new topic in 
maths straightaway, 1 know 
that I will never really 
understand it 


11 




5 


11 


16 


26 


34 


53 


45 


Maths is not one of my 
strengths 


8 




8 


26 


29 


34 


37 


32 


26 


1 learn things quickly in 

ntlip 


37 




26 


47 


58 


16 


13 


0 


3 



maths 



N = 38 38 

A series of single response items 

Due to rounding, percentages may not always sum to 100 
There were no non-responses to this question 
Source: NFER Year 7 Mathematics Survey 2005/6 



There was little change in Year 7 pupils’ views of enjoyment of lessons (Table 
3.5), with the great majority still finding mathematics ‘quite’ or ‘very’ 
enjoyable. 



Table 3.5 Year 7: Think about your recent maths lessons. Please tick the box 
which best describes this lesson. 



Level of enjoyment of most recent maths lesson 


Sweep 1 

% 


Sweep 2 

% 


Very enjoyable 


21 


21 


Quite enjoyable 


63 


61 


Not very enjoyable 


13 


13 


Not at all enjoyable 


3 


5 


N = 


38 


38 



A single response item 

Due to rounding, percentages may not sum to 100 
There were no non-responses to this question 
Source: NFER Year 7 Mathematics Survey 2005/6 
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Table 3.6 below shows that there was a marked increase in the percentage of 
secondary pupils who thought they had made ‘a great deal of progress’ in 
mathematics in the previous six months. The improved test scores for 
secondary Accelerated Maths schools reported above (in Chapter 2) would 
support pupils’ perceptions of progress. 



Table 3.6 Year 7: Perceived progress in maths over last six months 



Progress made in maths over the last six months 


Sweep 1 

% 


Sweep 2 

% 


A great deal of progress 


42 


63 


Some progress 


55 


34 


A little progress 


3 


3 


No progress 


0 


0 


N = 


38 


38 



A single response item 

Due to rounding, percentages may not sum to 100 
There were no non-responses to this question 
Source: NFER Year 4/5 Mathematics Survey 2005/6 

The survey included a question asking for pupils’ views about school in 
general. Amongst secondary pupils there was an increase in the proportions 
who agreed that they enjoyed being at school (from 63 to 74 per cent) and that 
a reason for enjoyment was getting on with others (from 53 to 68 per cent). 
There was greater uncertainty about whether work going well was a reason for 
enjoyment (Table 3.7). 

Table 3.7 Year 7: Views about school in general 



Level of agreement that... Agree Not sure Disagree No response 

% % % % 





SI 


S2 


SI 


S2 


SI 


S2 


SI 


S2 


I like the subjects that 1 know I can 
do well in 


79 


87 


11 


13 


11 


0 


0 


0 


When I enjoy my day at school it’s 
usually because I’ve got on well 
with the other pupils 


53 


68 


37 


26 


8 


5 


3 


0 


I enjoy being at school 


63 


74 


18 


18 


16 


8 


3 


0 


When I enjoy my day at school it’s 
usually because my school work has 
gone well 


76 


63 


18 


32 


5 


5 


0 


0 


N = 


38 


38 















A series of single response items 

Due to rounding, percentages may not always sum to 100 
Source: NFER Year 4/5 Mathematics Survey 2005/6 
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3.3 Mathematics: comparing primary schools 



In order to be able to compare attitudes across program and non-program 
schools, pupil questionnaire surveys were sent to a number of comparison 
schools as well as to pupils in treatment schools. In relation to mathematics 
there were six treatment schools (two secondary schools and four primary 
schools) and four comparison schools (one secondary and three primaries). 

Pupils in all of these schools completed both sweep 1 and sweep 2 of the 
questionnaire survey. The tables in this and the next section present 
comparisons of program and non-program school responses from the sweep 2 
survey: these findings are the most relevant because this second survey was 
completed at the end of the ‘treatment’ period, when any differences in 
attitudes between the pupils in the two school types would have been most 
evident. It was also useful, however, to look at changes between the sweep 1 
and sweep 2 findings, so as to look at the different starting points for the 
program and non-program samples, as well as the outcomes or end points 
towards the end of the school year. For this reason, any major differences in 
findings between sweep 1 and sweep 2 are also reported at the relevant points. 

The figures presented in Table 3.8 below show that Year 4/5 pupils in the 
comparison schools perceived themselves as doing better in mathematics, but 
in other respects pupil attitudes were more positive in the treatment schools. 
Here, more pupils agreed that they enjoyed learning mathematics, that they 
learnt more quickly and that they found their lessons interesting. There was 
also a higher percentage (54 per cent compared to 39 per cent) who agreed that 
they would l ik e to do more mathematics. 

With respect to the self-perception in relation to the first item in this table, ‘I 
usually do well in mathematics’, there were contradictory historical patterns 
across the treatment and comparison samples. In the treatment sample, 50 per 
cent of pupils in sweep 1 said that they usually did well in the subject, but by 
sweep 2 this proportion had declined to 44 per cent. In the comparison sample 
the reverse happened: only 40 per cent of pupils said that they did well in 
sweep 1, but this increased to 52 per cent by sweep 2. 

It is not really possible to identify any single cause for these trends, but one 
possibility is that the regular testing required in the Accelerated Maths scheme 
may have made pupils more self-critical about their progress in the subject. 
The comparison pupils (or teachers) may not have been ‘quantifying’ progress 
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in mathematics in quite the same way and may therefore have been more 
positive in their perceptions. 



Table 3.8 Years 4 and 5: Levels of agreement with statements about learning 
mathematics 



Views about 


Agree a 


Agree a 


Disagree a 


Disagree 




No 


mathematics 




lot 




little 




little 


a lot 


response 






% 




% 




% 


% 




% 


Sweep 2 


T 


C 


T 


C 


T 


C 


T C 


T 


C 


1 usually do well in 
maths 


44 


52 


42 


42 


7 


5 


2 1 


5 


1 


I would like to do 
more maths at school 

Maths is harder for me 


54 


39 


22 


33 


14 


16 


6 8 


4 


6 


than for many of my 
classmates 


10 


9 


22 


16 


25 


33 


37 36 


6 


6 


I enjoy learning maths 


65 


56 


20 


27 


6 


9 


3 2 


6 


6 


I am just not good at 
maths 


5 


9 


16 


10 


19 


19 


53 57 


6 


5 


I learn things quickly 
in maths 


44 


29 


31 


50 


14 


11 


5 3 


7 


7 


The work I do in 
maths lessons is 
interesting 


62 


49 


26 


33 


3 


9 


4 4 


5 


5 


N = 


219 


129 

















T = Treatment school; C = Comparison school 
A series of single response items 

Due to rounding, percentages may not always sum to 100 
Source: NFER Year 4/5 Mathematics Survey 2005/6 



As Table 3.9 shows, more than half the pupils from primary treatment schools 
described their most recent mathematics lesson as ‘very enjoyable’ (54 per 
cent), compared with 45 per cent in the comparison schools. Only two per cent 
of pupils, in either comparison or treatment schools, said that mathematics was 
‘not at all enjoyable’. 
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Table 3.9 Years 4 and 5: Level of enjoyment of most recent maths lessons 



Level of enjoyment of most recent maths lesson 
Sweep 2 


Treatment 

schools 

% 


Comparison 

schools 

% 


Very enjoyable 


54 


45 


Quite enjoyable 


36 


43 


Not very enjoyable 


5 


8 


Not at all enjoyable 


2 


2 


No response 


3 


3 


N = 


219 


129 



A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 4/5 Mathematics Survey 2005/6 



Although the pupils from program schools enjoyed their mathematics lessons 
more, a slightly greater proportion of respondents from non-program schools 
thought that they had made ‘a great deal of progress’ over the last six months 
(Table 3.10). The proportions of pupils expressing a view that they had made a 
great deal of progress increased for both samples between sweep 1 and sweep 
2, though the increase was greater for comparison pupils. As noted above, 
however, it is possible that the pupils from treatment schools had a more 
realistic view of their progress because they were tested more frequently as 
part of the program. 



Table 3.10 Years 4 and 5: Perceived progress in maths over last six months 



Progress made in maths over the last six months 
Sweep 2 


Treatment 

schools 

% 


Comparison 

schools 

% 


A great deal of progress 


46 


52 


Some progress 


43 


40 


A little progress 


6 


5 


No progress 


0 


1 


No response 


4 


2 


N = 


219 


129 



A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 4/5 Mathematics Survey 2005/6 



3.4 Mathematics: comparing secondary schools 

Generally, attitudes towards mathematics among the secondary school pupils 
was more positive in the comparison schools, except that a greater proportion 
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of pupils in the treatment schools agreed ‘a lot’ that they would l ik e to spend 
more time studying mathematics (Table 3.11). 



Table 3.11 Year 7: How much do you agree with these statements about 
learning maths? 



Views about 
mathematics 


Agree a lot 

% 


Agree a little 

% 


Disagree a 
little 

% 


Disagree a 
lot 

% 


No 

response 

% 


Sweep 2 


T 


C 


T 


C 


T 


C 


T 


C 


T 


C 


I usually do well in 
maths 

I would like to spend 


35 


64 


51 


34 


11 


2 


0 


0 


4 


0 


more time studying 
maths at school 

Maths is more difficult 


21 


9 


32 


55 


30 


27 


14 


9 


4 


0 


for me than for many 
of my classmates 


16 


7 


16 


18 


32 


34 


37 


41 


0 


0 


I enjoy learning maths 

Sometimes, when I do 
not understand a new 


30 


34 


47 


48 


14 


14 


7 


5 


2 


0 


topic in maths 
straightaway, I know 
that 1 will never really 
understand it 


16 


5 


18 


14 


26 


34 


37 


48 


4 


0 


Maths is not one of my 
strengths 


16 


9 


26 


27 


25 


34 


30 


30 


4 


0 


I learn things quickly 
in maths 


25 


25 


37 


61 


25 


11 


12 


2 


2 


0 


N = 


57 


44 



















A series of single response items 

Due to rounding , percentages may not always sum to 100 
Source: NFER Year 7 Mathematics Survey 2005/6 



Treatment school pupils were more prepared to describe their most recent 
mathematics lesson as very enjoyable, but more had also found it not very 
enjoyable (Table 3.12). 
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Table 3.12 Year 7: Think about your recent maths lessons. Please tick the box 
which best describes this lesson. 



Level of enjoyment of most recent maths lesson 


Treatment 

schools 


Comparison 

Schools 




% 


% 


Sweep 2 


Very enjoyable 


26 


20 


Quite enjoyable 


49 


61 


Not very enjoyable 


23 


14 


Not at all enjoyable 


2 


5 


N = 


57 


44 


A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 7 Mathematics Surx’ey 2005/6 






Comparison school pupils perceived their recent progress 


in mathematics 


more positively than the treatment school pupils, although these percentages 


are based on quite small numbers (Table 3.13). 






Table 3.13 Year 7: Perceived progress in maths over last six months 


Progress made in maths over the last six months 
Sweep 2 


Treatment 

schools 

% 


Comparison 

Schools 

% 


A great deal of progress 


39 


64 


Some progress 


53 


34 


A little progress 


9 


2 


No progress 


0 


0 


N = 


57 


44 



A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 7 Mathematics Survey 2005/6 



Summary 

The use of the pupil questionnaire survey allowed the research team to track 
any changes in pupil attitudes towards mathematics over the course of the 
school year 2005-2006. 

With respect to the primary pupils, overall, there was very little evidence of 
changed attitudes towards mathematics, towards school in general, or towards 
ways of working, over this time period. There were only a few exceptions to 
this general pattern. One of these was an important positive finding, relating 
to progress made in mathematics: the proportion of pupils reporting ‘a great 
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deal of progress’ made in the last six months increased from 41 per cent to 48 
per cent. 

In analysing the findings from the mathematics survey of secondary pupils 
more caution needs to be exercised because of the smaller numbers involved. 
That said, the perspectives of these Year 7 respondents generally seem to have 
become more positive than those of their Year 4/5 counterparts. Examples of 
positive developments include the following: 

• the proportion of pupils expressing a view that they usually did well in 
mathematics increased from 50 per cent to 61 per cent 

• the proportion of pupils reporting making ‘a great deal of progress’ 
increased from 42 per cent to 63 per cent 

• the proportion of pupils who ‘enjoyed being at school’ increased from 63 
to 74 per cent 

To some extent these findings match with those from the analysis of the test 
scores (see Chapter 2). Over the period in question both primary and 
secondary pupils using Accelerated Maths experienced improved average 
standardised test scores, but there was a larger improvement for the secondary 
sample. 

Overall, the questionnaire survey findings do not provide evidence of any 
significant differences in pupil attitudes to mathematics across treatment and 
comparison schools. For some questions the comparison pupil sample has 
slightly more positive attitudes and for others the treatment pupils have a 
slightly more positive outlook. These differences may reflect the influence of a 
number of factors, including teacher styles, quality of mathematics 
departments within the school, school culture and parental attitudes, as well as 
Renaissance versus non- Renaissance modes of delivery of mathematics. 
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4. Pupil attitudes: reading 



This chapter examines the findings from the pupil attitude questionnaire 
relating to reading. Sections 4.1 and 4.2 look at changes in pupil attitudes 
over time by comparing sweep 1 and sweep 2 findings from all the sample 
schools, dealing with primary schools and secondary schools respectively. 
Sections 4.3 and 4.4 compare attitudes to reading in program and non-program 
schools. 



4.1 Reading: progress over time - primary schools 

Attitudes to reading, as shown in the responses in Table 4.1 below, were 
positive, with the majority of Year 4/5 pupils (in sweep 2) agreeing that they 
enjoyed reading and did not find it difficult. However, there was a downward 
trend between the two sweeps in the percentage that agreed that they liked 
reading stories (from 76 to 70 per cent) and generally enjoyed reading (from 
70 to 64 per cent). Most pupils disagreed that reading was boring, although 
there was very little change in attitude between the two sweeps. 

The majority disagreed with the statement that they were not interested in 
books, but more agreed than disagreed that they liked watching TV better than 
reading and the trend here was slightly up. There was a downward trend in the 
percentage of pupils who liked reading with an adult (from 41 to 32 per cent) 
and this may have been related to an increase in self-sufficiency encouraged 
by the reading program, or it could have been related simply to the increase in 
the age of respondents between the two sweeps. 
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Table 4.1 Years 4 and 5: Levels of agreement with statements about reading 



Views about reading Agree Not sure Disagree No 

% % % response 





SI 


S2 


SI 


S2 


SI 


S2 


SI 


S2 


I like reading stories 


76 


70 


9 


18 


9 


11 


6 


1 


I am not interested in books 


17 


10 


14 


23 


62 


64 


7 


3 


I like reading comics or 
magazines 


63 


65 


17 


18 


14 


14 


7 


3 


I like reading poems 


57 


56 


18 


23 


16 


17 


8 


3 


I think reading is difficult 


20 


14 


27 


27 


45 


54 


8 


5 


I like reading silently by myself 


68 


64 


16 


15 


9 


16 


7 


4 


I like watching television better 
than reading books 


41 


42 


23 


28 


32 


26 


5 


4 


I like reading with a grown-up 
to help me 


41 


32 


19 


21 


34 


41 


7 


6 


I enjoy reading 


70 


64 


12 


18 


10 


13 


8 


4 


I think reading is boring 


15 


15 


13 


15 


64 


65 


7 


5 


N = 


271 


271 















A series of single response items 

Due to rounding, percentages may not always sum to 100 
Source: NFER Year 4/5 Reading Survey 2005/6 



While the percentage of pupils claiming to read every day at home decreased 
from 30 percent to 22 percent, there was an increase in those who said that 
they read most days and no change in those who said that they did not often 
read at home (see Table 4.2). 



Table 4.2 Years 4 and 5: Frequency of reading at home 



How often do you read at home? 


Sweep 1 

% 


Sweep 2 

% 


Every day 


30 


22 


Most days 


44 


48 


Not often 


19 


19 


Never 


4 


5 


No response 


4 


5 


N = 


271 


271 



A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 4/5 Reading Survey 2005/6 



Table 4.3 below shows that there was a slightly negative trend in reported 
levels of enjoyment of lessons involving reading, although this was based on 
the most recent individual lesson, rather than in general. 
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Table 4.3 Years 4 and 5: Level of enjoyment of most recent lesson involving 
reading 



Level of enjoyment of most recent reading lesson 


Sweep 1 

% 


Sweep 2 

% 


Very enjoyable 


49 


44 


Quite enjoyable 


30 


35 


Not very enjoyable 


11 


12 


Not at all enjoyable 


4 


7 


No response 


6 


3 


N = 


271 


271 


A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 4/5 Reading Survey 2005/6 






Views on progress in reading, however, 


were positive, with 


an increase in 


those saying that they had made ‘a great deal of progress’ from 46 to 52 per 


cent (Table 4.4). 






Table 4.4 Years 4 and 5: Perceived progress in reading over last six months 


Progress made in reading over the last six months 


Sweep 1 

% 


Sweep 2 

% 


A great deal of progress 


46 


52 


Some progress 


37 


34 


A little progress 


10 


8 


No progress 


3 


3 


No reponse 


4 


3 


N = 


271 


271 



A single response item 

Due to rounding , percentages may not sum to 100 
Source: NFER Year 4/5 Reading Survey 2005/6 



Levels of confidence in using computers showed a very positive development, 
with an increase from 56 to 66 per cent in those feeling very confident and a 
decrease in those who felt worried and who only ‘got by’ in their use (Table 
4.5). Although this may not all have been attributable to the use of the reading 
program, there is a strong possibility that regular use of the program 
contributed to this increase in confidence. 
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Table 4.5 Years 4 and 5: Level of confidence about using computers. 



Level of confidence about using computers 


Sweep 1 

% 


Sweep 2 

% 


I feel very confident 


56 


66 


I feel quite confident 


15 


15 


I feel 1 get by 


7 


3 


I feel a bit unsure about using computers 


6 


6 


I feel worried about using computers 


4 


1 


No response 


11 


9 


N = 


271 


271 



A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 4/5 Reading Survey 2005/6 



4.2 Reading: progress over time - secondary schools 

The secondary school pupils gave rather mixed responses to the general 
question on their views on reading (see Table 4.6 below). There was a decline 
in the numbers who liked reading stories and who said that they enjoyed 
reading and an increase in those who thought that reading was boring (from 9 
to 14 per cent). However, there was also a decrease in the percentage who 
agreed that they were not interested in books (from 16 to 8 per cent). 

There was a negative trend among the secondary pupils on reading at home, 
with the minority indicating that they read everyday staying the same at 17 per 
cent, but a decline from 52 to 35 per cent of those who read most days and an 
increase from 27 to 40 per cent of those who said that they did not read very 
often (Table 4.7). 
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Table 4.6 Year 7: Levels of agreement with statements about reading 



Views about reading 


Agree 


Not sure 


Disagree 


No 

response 






% 


% 






% 




% 




SI 


S2 


SI 


S2 


SI 


S2 


SI 


S2 


I like reading stories 


74 


65 


13 


21 


13 


12 


0 


2 


I am not interested in books 


16 


8 


20 


24 


63 


66 


1 


2 


I like reading comics or 
magazines 


73 


74 


18 


19 


8 


5 


1 


2 


I like reading poems 


56 


48 


28 


35 


16 


15 


1 


2 


I think reading is difficult 


8 


6 


21 


23 


70 


69 


0 


3 


I like reading silently by myself 


77 


66 


13 


24 


9 


6 


1 


5 


I like watching television better 
than reading books 


55 


53 


31 


36 


14 


9 


0 


2 


I like reading with a grown-up to 
help me 


18 


12 


19 


19 


63 


66 


1 


3 


I enjoy reading 


71 


59 


15 


28 


11 


10 


3 


3 


I think reading is boring 


9 


14 


19 


23 


71 


61 


1 


2 


N = 


108 


108 















A series of single response items 

Due to rounding, percentages may not always sum to 100 
Source: NFER Year 7 Reading Survey 2005/6 



Table 4.7 Year 7: Frequency of reading at home 


How often do you read at home? 


Sweep 1 


Sweep 2 


% 


% 


Every day 


17 


17 


Most days 


52 


35 


Not often 


27 


40 


Never 


4 


6 


No response 


1 


2 


N = 


108 


108 


A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 7 Reading Survey 2005/6 



There was also a negative trend in levels of enjoyment of a recent lesson 
involving reading, with a decrease from 29 to 23 per cent of those who had 
found it ‘very enjoyable’ and from 56 to 50 per cent of those who had found it 
‘quite enjoyable’ (Table 4.8). Twice as many pupils claimed to have found the 
lesson ‘not very enjoyable’ by the time of the second survey, although they 
were still the minority. As with the primary schools, it may be that the 
response was affected by the reference to the most recent lesson, rather than to 
lessons involving reading in general. 
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Table 4.8 Year 7: Level of enjoyment of most recent lesson involving reading 



Level of enjoyment of most recent reading lesson 


Sweep 1 

% 


Sweep 2 

% 


Very enjoyable 


29 


23 


Quite enjoyable 


56 


50 


Not very enjoyable 


9 


18 


Not at all enjoyable 


6 


7 


No response 


0 


2 


N = 


108 


108 


A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 7 Reading Survey 2005/6 






There was a decreased perception of having made ‘a great deal of progress’ in 
reading among the secondary pupils than had been the case with the primary 


respondents (Table 4.9). There was an increase 


in those who considered that 


they had made some progress (from 48 to 55 per cent), but a decrease from 43 


to 34 per cent of those who reported ‘a great deal’ of progress. 




Table 4.9 Year 7: Perceived progress in reading over last six months 


Progress made in reading over the last six months 


Sweep 1 

% 


Sweep 2 

% 


A great deal of progress 


43 


34 


Some progress 


48 


55 


A little progress 


8 


8 


No progress 


1 


1 


No response 


0 


2 


N = 


108 


108 



A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 7 Reading Survey 2005/6 



As with the primary pupils, there was a very positive trend in levels of 
confidence about using computers, with a five per cent increase in those who 
felt very confident and a decline in the small numbers who felt unsure or 
worried about using them (Table 4.10). 
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Table 4.10 Year 7: Level of confidence about using computers 



Level of confidence about using computers 


Sweep 1 

% 


Sweep 2 

% 


I feel very confident 


68 


73 


I feel quite confident 


21 


17 


I feel I get by 


6 


7 


I feel a bit unsure about using computers 


3 


1 


I feel worried about using computers 


1 


0 


No response 


1 


2 


N = 


108 


108 



A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 7 Reading Survey 2005/6 



Summary 

Primary school respondents’ attitudes towards reading were very mixed by the 
time of sweep 2 of the survey. In some instances attitudes seem to have 
become slightly more negative: for example, the percentage stating that they 
enjoyed reading declined from 70 to 64 per cent. However, even though levels 
of enjoyment seem to have decreased, there was evidence that pupils were still 
prepared to work hard and felt that they were making progress with their 
reading. For example, the proportion expressing the view that they had made 
‘a great deal of progress’ with their reading increased from 46 to 52 per cent. 

Overall the primary school responses to the survey show the development of 
rather more positive attitudes to English/reading lessons than those for the 
secondary schools, although in certain areas, such as confidence in using 
computers, there was a strongly positive trend across both age groups. 



4.3 Reading: comparing primary schools 



The majority of the following tables and commentaries are based on the 
second sweep of the pupil survey on attitudes to reading, received from a total 
of eight primary schools. 11 With respect to the first set of questions, based on 
statements about reading, the views of the treatment school pupils were 



1 1 As with the findings for mathematics, presented in Chapter 3, the tables in this and the next section 
present comparisons of program and non-program school responses from the sweep 2 survey, but 
in addition, any major differences in findings between sweep 1 and sweep 2 are also reported 
where these are deemed to be relevant. 
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considerably more positive than those of the comparison pupils (Table 4.11). 
Not only did higher percentages of treatment school pupils agree that they 
liked reading stories and enjoyed reading generally (73 per cent compared to 
63 per cent and 69 per cent compared to 53 per cent), but fewer of them 
agreed that they l ik ed watching television better (38 per cent compared to 53 
per cent). 



Table 4.11 Years 4 and 5: Levels of agreement with statements about reading 



Views about 
reading 


Agree 

% 


Not sure 

% 


Disagree 

% 


No response 

% 


Sweep 2 


T 


C 


T 


C 


T 


C 


T 


C 


I like reading 
stories 

I am not 


73 


63 


16 


19 


9 


17 


2 


2 


interested in 
books 

I like reading 


8 


14 


21 


34 


67 


50 


3 


2 


comics or 
magazines 


63 


73 


20 


13 


14 


13 


3 


2 


I like reading 
poems 


56 


58 


22 


27 


17 


16 


4 




I think reading is 
difficult 


14 


17 


29 


17 


51 


64 


6 


2 


I like reading 
silently by myself 

I like watching 
television better 


64 


69 


16 


16 


16 


13 


4 


3 


than reading 
books 

I like reading 


38 


53 


28 


27 


29 


19 


5 


2 


with a grown-up 
to help me 


34 


31 


22 


20 


38 


47 


6 


2 


I enjoy reading 


69 


53 


16 


23 


10 


20 


4 


3 


I think reading is 
boring 


12 


25 


16 


13 


66 


59 


6 


3 


N = 


232 


64 















A series of single response items 

Due to rounding, percentages may not always sum to 100 
Source: NFER Year 4/5 Reading Survey 2005/6 



The treatment school pupils’ response to a question about how often they read 
at home was also more positive than that of the comparison pupils, with a 
quarter of the respondents claiming that they read every day, compared to 14 
per cent of the comparison pupils (Table 4.12). 
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Table 4.12 Years 4 and 5: Frequency of reading at home 



How often do you read at home? 
Sweep 2 


Treatment 

Schools 

% 


Comparison 

Schools 

% 


Every day 


25 


14 


Most days 


47 


55 


Not often 


18 


20 


Never 


4 


8 


No response 


6 


3 


N = 


232 


64 



A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 4/5 Reading Survey 2005/6 



Nearly half the respondents from the treatment schools had found their most 
recent lesson very enjoyable, compared to 38 per cent from the comparison 
schools (Table 4.13). On the negative responses (‘not very enjoyable’ and ‘not 
at all enjoyable’), there was little difference. 



Table 4.13 Years 4 and 5: Level of enjoyment of most recent lesson involving 
reading 



Level of enjoyment of most recent reading lesson 
Sweep 2 


Treatment 

Schools 

% 


Comparison 

Schools 

% 


Very enjoyable 


47 


38 


Quite enjoyable 


31 


45 


Not very enjoyable 


12 


9 


Not at all enjoyable 


6 


8 


No Response 


3 




N = 


232 


64 



A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 4/5 Reading Survey 2005/6 



Comparison of sweep 1 and sweep 2 findings for the questionnaire items 
reported in Tables 4.12 and 4.13 suggest that the Accelerated Reader 
experience has provider a strong motivator for Year 4/5 pupils: whilst the 
proportion of treatment school pupils reading ‘every day’ dipped slightly 
(from 29 per cent to 25 per cent), the equivalent finding for comparison pupils 
indicated a significant decline in reading every day (from 35 to 14 per cent). 

There was a similar pattern in the proportions of pupils finding the most recent 
reading lesson ‘very enjoyable’ (for treatment pupils this dipped slightly from 
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49 per cent to 47 per cent; and for comparison pupils the decline was much 
more significant, from 53 to 38 per cent). It seems that involvement with the 
Accelerated Reader program may have helped to stem a decline in reading 
motivation across the school year. 

The figures given in Table 4.14 below show that pupil perceptions of how 
much progress they had made did not differ greatly, although more from the 
treatment schools described making ‘a great deal of progress’ (53 per cent 
compared to 48 per cent). 



Table 4.14 Years 4 and 5: Perceived progress in reading over last six months 



Progress made in reading over the last six months 
Sweep 2 


Treatment 
Schools % 


Comparison 
Schools % 


A great deal of progress 


53 


48 


Some progress 


33 


39 


A little progress 


8 


8 


No progress 


2 


5 


No Response 


3 




N = 


232 


64 



A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 4/5 Reading Survey 2005/6 



4.4 Reading: comparing secondary schools 

The responses set out below are also based on the second wave of the pupil 
survey on reading, administered towards the end of Year 7. Three schools 
returned the survey, two were treatment schools and one was a comparison 
school. 

Overall, secondary pupils’ general views on reading were much more positive 
in the treatment schools than in the comparison schools (Table 4.15). The 
only areas where there were higher levels of agreement from the comparison 
pupils were that they liked reading comics, magazines and poetry. Reading 
with an adult had a negative response from pupils in both types of school. 
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Table 4.15 Year 7: Levels of agreement with statements about reading 



Views about 
reading 


Agree 

% 


Not sure 

% 


Disagree 

% 


No response 

% 


Sweep 2 


T 


C 


T 


C 


T 


C 


T 


C 


I like reading 
stories 


72 


46 


16 


36 


10 


18 


2 


0 


I am not 
interested in 
books 


7 


23 


26 


23 


65 


54 


2 


0 


I like reading 
comics or 


72 


82 


18 


15 


7 


3 


2 


0 


magazines 

I like reading 
poems 


49 


54 


33 


36 


16 


8 


2 


3 


I think reading is 
difficult 


5 


8 


17 


33 


74 


59 


3 


0 


I like reading 
silently by myself 


68 


59 


20 


33 


5 


8 


6 


0 


I like watching 
television better 


51 


69 


35 


28 


12 


72 




0 


than reading 
books 


J 


z 


I like reading 
with a grown-up 


12 


15 


17 


23 


68 


62 


3 


0 


I enjoy reading 


64 


46 


23 


36 


10 


18 


3 


0 


I think reading is 
boring 


12 


23 


24 


26 


62 


51 


2 


0 


N = 


94 


39 















A series of single response items 

Due to rounding, percentages may not always sum to 100 
Source: NFER Year 7 Reading Survey 2005/6 



Interestingly, comparison of sweep 1 and sweep 2 responses revealed that 
comparison pupils had become much more interested in television over this 
period (see item 7 in Table 4.15), whereas treatment pupils had developed less 
of a liking for television (though in both sweeps and in both samples more 
than 50 per cent l ik ed watching television better than reading books). In 
sweep 1 just over half of the comparison sample (53 per cent) l ik ed television 
better than reading books, but this had increased to over two-thirds (69 per 
cent) by sweep 2. The equivalent figures for the treatment sample were 56 per 
cent in sweep 1, declining to 51 per cent in sweep 2. 



The percentage of pupils saying that they did not often read at home was quite 
high from both types of school, but higher in the comparison school (46 per 
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cent compared to 38 per cent) (Table 4.16). More treatment school pupils (19 
per cent) read at home every day than comparison school pupils (3 per cent). 



Table 4.16 Year 7: Frequency of reading at home 



How often do you read at home? 
Sweep 2 


Treatment 

Schools 

% 


Comparison 

Schools 

% 


Every day 


19 


3 


Most days 


34 


46 


Not often 


38 


46 


Never 


5 


5 


No Response 


3 




N = 


94 


39 



A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 7 Reading Survey 2005/6 



Treatment school pupils were more positive about their recent lessons, with 
more describing them as ‘very enjoyable’ (27 per cent compared to 15 per 
cent) and fewer saying that they were not very or ‘not at all enjoyable’ (Table 
4.17). 



Table 4.17 Year 7: Level of enjoyment of most recent lesson involving reading 



Level of enjoyment of most recent reading lesson 
Sweep 2 


Treatment 

Schools 

% 


Comparison 

Schools 

% 


Very enjoyable 


27 


15 


Quite enjoyable 


52 


41 


Not very enjoyable 


14 


36 


Not at all enjoyable 


5 


8 


No Response 


2 




N = 


94 


39 



A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 7 Reading Survey 2005/6 



As with the primary pupil responses, comparison of sweep 1 and sweep 2 
findings for the questionnaire items reported in Tables 4.16 and 4.17 suggest 
that the Accelerated Reader experience has provider a strong motivator for the 
Year 7 pupils: whilst the proportion of treatment school pupils reading ‘every 
day’ increased (from 16 per cent to 19 per cent), the equivalent finding for 
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comparison pupils indicated a significant decline in reading every day (from 
15 per cent to 3 per cent). 

There was a similar pattern in the proportions of Year 7 pupils finding the 
most recent reading lesson ‘very enjoyable’ (for treatment pupils this dipped 
slightly from 29 per cent to 27 per cent; and for comparison pupils the decline 
was much more significant, from 38 to 15 per cent). It seems that, as with the 
younger pupils, involvement with the Accelerated Reader program may have 
helped to stem a decline in reading motivation across the school year. 

The percentages given in Table 4.18 show that the majority of pupils in both 
types of school described their progress in reading as modest, but there was a 
higher percentage in the treatment schools who believed that they had made ‘a 
great deal of progress’(38 per cent compared to 28 per cent). 



Table 4.18 Year 7: Perceived progress in reading over last six months 



Progress made in reading over the last six months 
Sweep 2 


Treatment 

Schools 

% 


Comparison 

Schools 

% 


A great deal of progress 


38 


28 


Some progress 


51 


64 


A little progress 


9 


5 


No progress 




3 


No Response 


2 




N = 


94 


39 



A single response item 

Due to rounding, percentages may not sum to 100 
Source: NFER Year 7 Reading Survey 2005/6 



Summary 

Pupil responses on reading were generally more positive from the treatment 
schools at both primary and secondary level, with greater enthusiasm for 
reading, better motivation and a more developed sense of having made 
progress in this area. 
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5 



The teacher perspective 



5.1 The experience of using Renaissance Learning 

In order to collect complementary qualitative data, the research team visited 
four case-study schools (primary mathematics, primary reading, secondary 
mathematics, secondary reading) and interviewed two teachers involved with 
the programs in each school. 12 The reading program was used in the primary 
school as part of a guided reading programme with both year 4 classes and in 
the secondary school by the whole of year 7. The mathematics program was 
used by the primary school in special sessions, as an addition to normal maths 
lessons, and in the secondary school by two groups in two out of four of their 
weekly maths lessons. 

In three schools, the Renaissance Learning program was described as being 
additional to existing activities. One of the teachers interviewed commented 
that their numeracy strategy had not changed and it would only be possible for 
the mathematics program to replace existing work ‘if the objectives from the 
program could be matched up with the National Curriculum’. 13 She added that 
this could be done, but would be ‘time-consuming’. In the remaining school (a 
secondary school), the Accelerated Reading program had largely replaced 
previous reading strategies, because it was regarded as more suitable for 
individualised learning and offered more information about individual 
progress and skills than the existing programmes. 

In all the schools the staff interviewees had a positive attitude by about using 
the Renaissance Learning programs. In the primary school using the 
mathematics program, there was seen to be a particular advantage in ‘the fact 
that the computer does the marking, which frees teachers to deal with pupils 
who are having difficulties’. The only drawback to using the programs took 
the form of occasional technical problems. These were an irritation, because, 



12 In one school, one of the teachers involved in implementing the program had left, and only one 
other teacher remained involved with using the program, so a total of seven teachers rather than 
eight were interviewed. 

13 In fact, by the time the research was being carried out. Renaissance Learning had matched the 
Accelerated Mathematics objectives to those of the UK National Curriculum, but the interviewee 
was clearly not aware of this. 
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as one teacher pointed out, ‘it takes time to solve these problems’, even with 
technical support. The consequences of technical unpredictability were 
described by another interviewee: ‘If a pupil is ready to do the questionnaire 
and then can’t for some reason, they are stuck’. 



5.2 Links with pupils and parents 

The responses to a question about whether pupils l ik ed using Renaissance 
Learning programs were strongly positive in all four schools. Staff in the 
primary school using the mathematics program, explained how, ‘they often 
ask to do Renaissance Learning work and are disappointed if they can’t. They 
l ik e using computers and they l ik e the instant feedback from the tests’. The 
interviewee in the secondary school using the reading program commented: 
‘They definitely enjoy using it - if they lose their pass words, they can get 
upset because they can’t access it’. 

The point about the appeal of using computers and the ‘instant feedback’ they 
gave was repeated in all the schools and was reflected in this view: ‘The 
students l ik e it because they get feedback, they get reports and it makes them 
independent’. The element of competition was also referred to by those using 
both the mathematics and the reading programs and was seen a motivating 
factor: ‘They l ik e the quizzes and many like the element of competition in 
getting good scores’. In both the schools using the reading program, there was 
a perception that boys especially had been encouraged to read more, with the 
primary teacher stating that, ‘some boys have definitely tackled some fiction 
that they would probably not have touched before’ . 

In two of the schools (one primary mathematics and one secondary reading), 
staff felt that using the Renaissance program had strengthened links with 
parents. In the primary school, the program had helped to involve parents 
more closely with their children’s work: 

The marked tests go home and so parents get to see how their children 
are doing and they are given directions to help the children with their 
work. Parents do come in and talk about their children ’s progress and 
what they are doing. 

In the secondary school, the use of the reading program was reported as 
having made an important contribution to the school’s family learning 
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programme. The school had many pupils from recently arrived immigrant 
families and had an outreach centre that offered parents courses in English, 
mathematics and IT. Work on building links with Somali and Roma families 
had been particularly useful, not only in assisting the children with 
mathematics and reading, because ‘the parents now understand why their 
children bring books home and they can support them’, but also the parents 
themselves were learning English and had a better understanding of how the 
education system worked, so, for example, they realised why it was necessary 
for their children to attend school regularly. A special section for parents was 
being set up in the school library to encourage the further development of 
family reading. 



5.3 Issues with the programs 

Interviewees in all four schools said that they had been faced with 
technological problems when they first started using the programs. Issues such 
as ‘constant crashing’, ‘the printer not working’, an unreliable pass word 
system and ‘some answers were correct, but the software flagged them up as 
wrong’, had caused irritation. At this early stage, technical support from 
Renaissance Learning was not always available, so schools had to rely on their 
own technical knowledge or other local support, and in one school it was 
claimed that they were given ‘bad advice, because the adviser from 
Renaissance Learning was not properly trained’. This situation had improved 
considerably after a time, so that all the schools agreed that ‘It is better now as 
I have a telephone link with Renaissance Learning. They are helpful in that 
way’. There was only one non-technical problem, reported by one school, 
which was that: 

Most pupils were at level 3, but we did not have enough books for them 
and very little that was suitable for EAL students, so we had to give 
them simple books to read and then make up our own questions. 

However, this issue had also been dealt with as ‘Renaissance Learning have 
provided us with a lot of books at level 3 to 4 and below, so it’s much better’. 

When they were asked about their perceptions of general weaknesses in the 
programs, all the interviewees referred to the issue of technical dependency, 
which meant that ‘if one small thing goes wrong, the whole lesson falls apart’. 
One of the schools commented on the high level of support they had from 
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Renaissance Learning and the concern that when the pilot ended and they had 
less help, that it ‘will be all right’. One school also commented on the 
‘enormous amount of paper, ink and toner’ that they used, because ‘the maths 
program is paper-based’. 

Both the schools using the reading program had realised that ‘the teacher has 
to be very aware of how the pupils are using it’. In the primary school, it was 
reported that care had to be taken in interpreting the test scores, because ‘if 
some are coasting, they can appear to have very good test results, but aren’t 
actually reading as much as some others’. In the secondary school, they had to 
‘log which pages they [the pupils] had read to stop them from cheating and 
then reward them when they have met their targets’, as ‘some were doing the 
quiz questions without reading the books’. 

The extent of the staff training necessary to run the programs was referred to 
by interviewees in all the schools and the need for a ‘fair bit of commitment 
from one or two teachers to start with, in order to get really familiar with it’. If 
a school had high staff turnover, as one in particular did, this could cause 
problems. One school had been left with only one trained teacher and the 
librarian ‘holding it all together’, although Renaissance Learning had assisted 
by sending a seconded teacher to help out. 

In the secondary school using the mathematics program, one interviewee felt 
that the program was not for use with lower ability groups, but was ‘designed 
for the upper groups, although they [Renaissance Learning] don’t make this 
clear ’. 14 An interviewee in the primary mathematics school commented on the 
difficulty of ‘matching the content to the National Curriculum’ and the fact 
that the program ‘does not prepare pupils for Key Stage tests, which are a very 
different format’. In general, however, the schools accepted that the programs 
were not ‘a cure-all’ and by adaptation and addition to other schemes, they had 
a valuable part to play in developing numeracy and literacy. 



14 In fact Accelerated Maths is designed to be appropriate for pupils at all ability levels. Teachers are 
expected to use mathematics achievement tests (such as STAR Maths) and their professional 
judgement to decide the grade level of the Accelerated Maths library a pupil should work with. 
Low-achieving pupils can be placed in a lower library and are expected to be given extra tuition on 
the relevant mathematics topics. In addition. Accelerated Maths professional development 
seminars cover methods which teachers can use to help struggling pupils. 
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5.4 Effectiveness, impact and benefits 



Staff in the two schools using the mathematics program agreed that it was a 
useful addition to the strategies they already used, for, as one teacher 
explained: ‘It does not replace the teaching, but enables teachers to pinpoint 
who needs help and on what topic’. The primary school respondent using the 
reading program felt it was helpful when used alongside discussion about the 
books, because ‘you can’t just let them read and do the tests’. 

The secondary reading school interviewees were particularly positive about 
the program’s contribution to personalised learning and its effectiveness in 
helping a situation where there was a huge variation in reading ages at the 
beginning of Year 7. Pupils could work on the program in their own time and 
many did, because ‘it has motivation built into it with the points system’ and 
this was reinforced with their own school awards. This school also built on the 
interest in reading encouraged by the program, by bringing writers in to speak 
to the pupils. 

The case-study visits took place before the second wave of tests were 
completed, so staff had no data on which to base their views on possible 
effects on attainment. They were however, asked for their perceptions on what 
the impact of the programs might be on attainment. In the primary 
mathematics school, teachers thought that pupils’ self-esteem in the subject 
had been raised and that their more positive attitude would affect their 
attainment. An interviewee at the secondary mathematics school said that she 
thought from the evidence of marking work, that the program was helping the 
pupils. In the secondary reading school, the evidence from internal tests 
suggested that the majority of pupils had made progress and that the program 
had made individual pupil assessment easier. 

All the interviewees agreed that one of the most significant strengths of the 
Renaissance Learning programs was that they motivated pupils. Pupils using 
both the mathematics and the reading programs responded well to being in 
charge of their own rate of progress and seeing that progress confirmed. From 
the teachers’ point of view too, one of the main advantages was the 
personalised learning aspect of the programs, which, in the words of one 
interviewee, ‘addresses the difficult issue of differentiating teaching and 
learning in very widely mixed ability classes’. In addition to their value for 
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individualised learning, the programs could also encourage co-operation. As a 
teacher in the primary reading school explained: 

the program works well with SEN children too - they can have a 
partner when they do the tests who can help read the questions, but 
they can answer if they have understood the books. 

This point was also taken up in the primary maths school, where the teacher 
described how the program: 

encourages children to help each other, plus they get to know how it 
works and actually help the teacher troubleshoot and solve problems 
that their classmates might be having. It is quite empowering for the 
children. 

From a practical point of view, the way the programs worked was seen in all 
the schools as aiding monitoring and target setting and therefore also 
reporting. In the primary mathematics school, it was pointed out that they ‘can 
print out the diagnostic tests so the teacher can see very clearly in which areas 
they are not getting on and need extra support’. It also meant that ‘the marked 
tests and scores go home with the children at the end of every week’. 
Similarly, in the secondary reading school, the interviewee explained that the 
program helped the school to ‘use support and intervention more effectively’, 
and that ‘the monitoring and intervention are real strengths’. In addition, both 
the reading schools felt that the scheme was a good way of testing 
comprehension and that it encouraged boys to read, because the books ‘cover 
topics that are likely to encourage interest’. 



Summary 

As a final question, the interviewees were asked if they would recommend the 
use of the RL program to colleagues in their own and other schools and all 
said that they would. This was qualified in one school by a point about the 
cost, not just of the program, but of installing extra computers and in another 
by a reference to a previously raised point about the mathematics program’s 
use with lower ability groups. In the other two schools, not only was the 
reaction positive, but the teachers involved had already begun promoting the 
program. In one case this was targeted at feeder primary schools and in the 
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other, the program’s use was to be extended to Years 3 and 5 and would 
probably encompass the reading program too. 

The schools’ reactions to this last question were a good indicator of the 
predominantly positive view of the Renaissance Learning programs. 
Adaptation to the ways in which each school works also emerged as 
important. The other general message from the teachers interviewed was that 
the programs’ use of IT was both a major strength in the individual flexibility 
it allowed, but also a potential weakness when it did not function effectively. 
Overall however, the advantages of use appeared, at this stage, to considerably 
outweigh any disadvantages. 
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6. Overview and recommendations 



This report has drawn from three sources of data in order to assess the 
effectiveness of the Renaissance Learning programs in improving pupil 
achievement and motivation in a number of Specialist and feeder schools in 
London. The findings relating to standardised tests in reading and mathematics 
have been reported in Chapter 2, the responses to the pupil surveys have been 
detailed in Chapters 3 and 4, and Chapter 5 outlined teacher perspectives 
based on face-to-face interviews. This final chapter pulls these findings 
together in order to provide an overview of the key issues and benefits arising 
from the use of the programs, and also offers some initial recommendations 
for consideration. 



6.1 Issues arising from the programs 

Whilst the test data indicated overall improvements in average pupil 
standardised scores in both mathematics and reading in the Renaissance 
schools, and the questionnaire data indicated mostly no decline in attitudes 
and, in some cases, improvements in pupil perceptions, the data also point to a 
number of issues that are worthy of consideration with respect to the future 
implementation of the programs. These issues were mostly raised in the 
teacher interviews (even though the teachers were predominantly positive 
about the programs) and can be categorised under three headings: technical 
issues; support and professional development; and targeting pupils. 

Technical issues 

The Accelerated Maths and Accelerated Reader programs were introduced 
into schools which were mostly operating in challenging, inner-city, socio- 
economic circumstances, and school staff had many demands operating upon 
them in what were inevitably very busy institutions. In this context, it was not 
surprising that some of the school staff complained that, during the 
introductory period of using the programs, technical issues could be irritating 
and time consuming. 
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As noted in the previous chapter, interviewees in all of the four schools visited 
said that they had encountered technological problems when they first started 
using the programs. These had included computers crashing and printers not 
working. In some cases the problems may have been general technological 
issues and nothing to do with the Renaissance Program, but because they were 
part of the package of implementation they were associated with the new 
programs. In addition, all the teachers mentioned the problem of ‘technical 
dependency’. The Renaissance Programs were very helpful in that they could 
be used to provide complete lessons with the setting and assessment of pupils’ 
work all in one package, but if anything went wrong then the whole lesson 
(and the follow up) would be affected. 

Support and professional development 

It is clear that the successful implementation of the programs required a high 
degree of commitment from the teachers responsible for the mathematics and 
English/reading groups who were using the programs. Sufficient support in the 
early stages of the implementation of the programs, and the appropriate 
ongoing professional development for those involved in using them (including 
training from schools themselves and possibly from local authorities or the 
Specialist Schools and Academies Trust), may be necessary for maximising 
the success of the programs. 

Teacher interviewees noted how, at the early stages of implementation, 
technical support from Renaissance Learning was not always available, so 
schools had to rely on their own technical knowledge or other local support, 
though this situation had improved considerably as the year went on. 
Additionally, the extent of staff training necessary to run the programs to 
maximum effect was referred to by interviewees in all the schools. 

Targeting pupils 

In Chapter 2 it was shown that the test results for secondary mathematics and 
primary reading improved more than those for primary mathematics and 
secondary reading. These findings lead to questions about policies for 
targeting pupils in these two subjects. Two of the teacher interviewees also 
raised questions about targeting pupils. As was noted in Chapter 5, one of the 
teachers felt that the secondary mathematics program was most appropriate for 
higher ability pupils or groups. Another felt that the Accelerated Reading 
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program was not linked closely enough with the National Curriculum 
targets. 15 



6.2 Benefits of the programs 



The views expressed by the teacher interviewees were made in a constructive 
fashion and it should be stressed that they were predominantly positive about 
the impact of the Renaissance Programs on their teaching and on their pupils’ 
achievements and attitudes. Three main benefits of using the programs can be 
identified from these comments and from the test and survey data: 

Improvements in attainment 

Bearing in mind all the qualifying comments made about sample sizes, the 
short timescale involved and the multiplicity of variables that can affect test 
scores (see Section 2.1) it does nevertheless need to be emphasised that there 
were improvements in average standardised test scores in the treatment 
schools for mathematics (both secondary and primary) and in the primary 
schools for reading. These were not spectacular improvements, but they can be 
seen as an important step in the right direction, progress that has occurred 
sometimes in schools where improvements are historically difficult to achieve 
because of the impact of external factors, such as population mobility, limited 
levels of parental support and high levels of socio-economic deprivation. 
Relevant key findings regarding the test data can be summarised as follows 
(see also Chapter 2): 

• Taking all the schools using Accelerated Maths , the average standardised 
pupil score (N=332 in sweep 1) increased by three points from 94.61 to 
97.61; in the comparison schools (not using the program) the average 
standardised pupil score (N=170) decreased from 100.40 to 99.84. 

• In primary schools using Accelerated Reader the average standardised 
pupil score (N=225 in sweep 1) increased marginally from 92.23 to 92.64; 
in the comparison primary schools (not using the program), however, the 
average standardised pupil score (N=66) also increased, and by a bigger 
margin. 

• Taking all the schools returning tests for this evaluation, in both 
mathematics and reading, it was found that in the 14 schools using 



15 Since the basic aim of the Accelerated Reader program is to promote reading (generally), it could 
be argued that it need not be linked directly to the National Curriculum. The spring 2007 release 
of Accelerated Reading, however, will include the capacity for the provision of details of pupils' 
Estimated Reading Age and their National Curriculum Level scores. 
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Renaissance Programs the average standardised pupil score (N=658 in 
sweep 1) increased by nearly half a point from 93.79 to 94.23, whereas in 
the seven comparison schools the average standardised pupil score 
(N=277) decreased very slightly, from 98.41 to 98.36. 

Improvements in attitudes 

In general, the questionnaire data suggest that, with a few exceptions, there 
were no significant changes in pupil attitudes towards their subject 
(mathematics or reading), towards school in general, or towards ways of 
working, over the course of the school year. The exceptions, however, were 
mainly positive and were largely to do with working hard and making progress 
(rather than intrinsic enjoyment of the subject). Increased proportions of both 
Year 4/5 reading pupils and Year 7 mathematics pupils, in program schools, 
for example, expressed a view that they had made ‘a great deal of progress’ in 
these subjects over the eight-month period in question: the increases were 
from 46 to 52 per cent for the former and from 42 to 63 per cent for latter. 

In addition, as noted previously, all the staff interviewees said that their pupils 
l ik ed using the programs, to the extent that they could be disappointed if they 
discovered that a particular lesson did not involve the Renaissance program. 
All the interviewees agreed that one of the main strengths of the programs was 
that they helped to motivate students. 

Flexible learning 

Teacher interviewees were also enthusiastic about the contribution of the 
programs, alongside other forms of classroom activity, to flexible and 
‘personalised’ learning. One respondent, for example, commented on the fact 
that the computer does the marking, ‘which frees teachers to deal with pupils 
who are having difficulties’. The point about the appeal and usefulness of 
‘instant feedback’ was repeatedly made by interviewees in all the schools. 
Additionally, in two of the schools, staff felt that using the program had 
strengthened links with parents, something which can be difficult to achieve in 
urban schools. The personalised learning aspects of the programs helped to 
address issues of differentiation for mixed ability classes, and of developing 
monitoring, intervention and support strategies. 



52 




6.3 Conclusions and recommendations 



This was the first year of full implementation of these programs in any UK 
schools and this was effectively a pilot year for this initiative. In this context 
the research team offers the following recommendations for consideration 
regarding the further implementation of these programs: 

• The main issues raised were to do with the use of technology in the very 
early stages of implementation, and all of these were subsequently 
resolved, but it may be worth giving further consideration to the question 

of how early technical problems can be avoided and quickly resolved. 

• Another issue worth considering further is that of whether there should be 
any emphasis on a particular program (mathematics or reading) or age 
group (Years 4/5 or 7). The evaluation revealed that schools using the 
secondary mathematics and primary reading programs were achieving the 
best results - what implications does this finding have for the future 
development of these programs in a UK context? It also seems that the 
question of ‘which age and ability groups would benefit most from using 
the programs?’ is one that is worth considering further. 

• The variations across program schools suggest that it would be useful to 
look at good practice case studies. For example, how were the two 
secondary mathematics schools able to achieve such positive test scores in 
just an eight-month period: is there anything that other schools could learn 
from the successes of these two schools? 



There is already some evidence of success. In combination, the findings from 
this evaluation suggest that the use of the Renaissance Learning programs has 
contributed to steady progress in most of the schools using them. In some 
schools, particularly those using Accelerated Maths with Year 7 groups, 
progress has been beyond the levels that might have been predicted. If such 
examples can be replicated then the Renaissance Learning Programs have 
much potential for improving numeracy and literacy levels in UK schools. 
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